Weak convergence on Douglas- Rachf ore! method 
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Abstract 
C*~) ' We prove that the sequences generate by the Douglas-Rachford 

method converge weakly to a solution of the inclusion problem 
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Prom now on H is a real Hilbert space with inner product (•, •) and 
associated norm || • ||. A point-to-set operator T : H z4 H is a relation 
T C H x H and for x G H, 

£2; T(x) = {veH\(x,v)eT}. 

fvi An operator T : H =$ H is monotone if 

C^; ' («l -»2,«l -^2) > 0,V(xi,?;i),(x2,W2) e T 

o ; 

and it is maximal monotone if it is monotone and maximal in the family of 
monotone operators with respect to the partial order of inclusion. 

Let A, B be maximal monotone operators in H. Consider the problem 
S^ , of finding x G H such that 

0£A(x) + B(x). (1) 

Douglas-Rachford method (with sumable residual error) works as follows. 
Let {«&}, {/3fc} be sequences of positive error tolerance such that 

00 00 

^Q fc <oo, y~]Pk < 00 . 
fe=i fc=i 
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Take A > 0, (x , b Q ) G B and for k = 1, 2, . . . 

(a) Find (y k ,a k ) such that 

(y k ,a k )EA, \\y k + \a k - (x k -i - \b k _i)\\ < a k (2) 

(b) Find (x k ,b k ) such that 

(x k ,b k )eB, \\x k + Xb k - (y k + Ab fc -i)|| < fa (3) 

Existence of (y k ,a k ), (x k ,b k ) as above follows from Minty's Theorem [6]. 
From now on {(x k , 6fc)}£l , {(j/fe, afc)}^! are sequences generated by the 
above algorithm. This algorithm was proposed by Douglas and Rachford [lj 
for the case where A and B are linear, and was extended to arbitrary max- 
imal monotone operators by Lions and Mercier [5]. These later authors 
proved that x k + \b k converges weakly to a point z such that, for some x, b, 

z = x + Xb, beB(x), -beA(x). 

Our aim is to prove the next theorem 

Theorem 1. If A,B are maximal monotone operators and the solution set 
of 

0£ A(x) + B(x) 

is non-empty, then the sequences {(x k , b k )} and{(y k ,a k )} generated by Douglas- 
Rachford method converges weakly to some (x, b) and (x, —b) respectively, 
such that 

beB{x), -bEA(x) 

and hence, G A{x) + B{x). 

For each k, there exists unique pairs (y k ,a k ), (x k ,b k ) such that 

{i)k, a k ) E A, y k + Xa k = x fe _i - A6 fc _i (4) 

(x k , b k ) G B, x k + \b k = y k + A6 fe _i (5) 

Existence of (y k ,a k ), (x k ,b k ) as above follows from Minty's Theorem [6]. 
Direct use of (jl|) and © shows that for k = 1, 2, . . . 

x k -i- x k = X(a k + b k ), \{b k _i - b k ) = x k - y k . (6) 

and 

x k -y k + \{a k + b k ) = \{a k + 6 fc _i) = x fc _i - y k . (7) 



Using (|2|) , (JH) and the monotonicity of A we conclude that 



\\Vk ~ Vk\\ 2 + A 2 ||a fc - dfell <\\yk ~Vk + A(a& - afc)|| 2 

= \\y k + Xa k - (xk-i - A6fe_i)|| 2 < a\ (8) 

Using ©, ((31), the monotonicity of B, triangle inequality and the above 
inequality we have 



\xk -x k \\ 2 + A 2 ||&fc 



< 



x k -x k + \(b k - b k ) 



< \\x k + \b k - (y k + A6 fc _i)|| 2 

< (Ikfc + A6 fc - (y k + A6 fe _i)|| + \\y k - y k \\) 2 
<(a k + h) 2 (9) 



We will use in H x H the inner product {-,-)\ and associated norm 



lA> 



{(x,v),(x',v')) x = (x,x') +X 2 (v,v'), \\p\\ x = \/{p,p) x . 

Note that HxH endowed with the inner product (•, -) x is a Hilbert space iso- 
morphic to HxH endowed with the canonical inner product {(x, v), (a/, v')) = 
(x,x') + (v,v'). Hence, the strong/weak topologies of both spaces are the 
same. To simplify the exposition, define 



Pk 



(x k ,b k ), A: = 1,2, 



p k = (x k ,b k ), A; = 0,1,..., 
We have just proved in ([9]) that 

\\Pk-Pk\\\<a k + f3 k , A; = 1,2,... 
The extended solution set [2J of problem (fTJ) is 

S(A, B) = BD-A = {(z, w) | (z, w) G B, (z, -w) G A} 
Lemma 2. If p £ S(A,B) then, 



(10) 



(11) 



?fc-i - p\\ 2 x > \\p k - p\\l + ||a;;fe-i - y fc || 2 



pIIa + A 2 ||dfc + b k -i\ 



fork = 1,2,. 



Proof. First note that p = (x,b) with b G -B(x) and —6 G A(x). Using the 
first equality in ([6|), the monotonicity of B and the monotonicity of A we 



conclude that 

(x k -i - x k ,x k - x) =X(a k + bk, x k - x) 



-X 



(a k + b, x k - x) + (bk -b,x k - x) 



>X(a k + b,x k - x) 



-X 



(a k + b,x k - y k ) + (a k + b,y k - x) 



> X(a k + b,x k -y k ) 



Using the above inequality, the second equality in ([6]) and (fTUj) we have 

(Pk-i ~Pk,Pk -p}\ =(xk-i ~x k ,x k -x) + A 2 (6 fc _i -b k ,b k -b) 
>X(a k + b,x k - y k ) + X(x k - y k , b k - b) 
=X(x k -y k ,a k + b k ) 

Using (fTUj) and ([6]) we have 

\\Pk-i -Pk\\l = A 2 ||a fe + b k \\ 2 + \\x k -y k \\ 2 



Therefore, 



bfc-l-p|lA 



■-\\p k -l -Pk\\l + 2(pjfc_i - p k ,Pk - p) x + WPk - P\\l 
>A 2 ||a fc + b k \\ 2 + \\x k - y k \\ 2 + 2X(x k - y k , a k + bk) 
+ \\Pk-p\\l 



= \\Pk ~ p\\ x + \\xk -Vk + A(6 fc + a k ) 
To end the proof, use the above inequality and ([7]). 
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Corollary 3. The sequence {p k } is Quasi-Fejer convergent to S(A,B). 
Therefore it has at most one weak cluster point in this set, and it is bounded 

Proof. Take p G S(A,B). Using (lllh and Lemma [2] we have 

\\Pk ~P\\ < \\Pk ~Pk\\ + \\Pk ~P\\ <a k + f3 k + \\pk-i -p\\ 

which proves that {p k } is Quasi-Fejer convergent to S\(A,B). The last part 
of the corollary follows from this result and also from Opial's Lemma [7]. □ 



Proof of Theorem [TJ We are assuming that problem ([I]) has a solution. There- 
fore, S(A,B) / 0. Using Corollary [3] we conclude that {p k } is bounded. 
Take 

peS(A,B) 

and let 

M = \ + sup||p fc -p|| A 

Using Lemma [2] we have, for k = 1, 2, . . . 

II* i|2^n m2 ii * ||2 

\\p k - p\\x < \\Pk-i -p\\ x - \\xk-i -y k \\ ■ 

Therefore, using also the concavity of t \— > \/t we conclude that 

\\p* k -p\\x < \\Pk-i -p\\ x -—\\x k - 1 -y k \\ 2 

Hence, combining the above inequality with (llip and triangle inequality we 
obtain 

\\Pk ~p\\x <\\Pk-Pk\\\ + \\Pk-p\\x 

<(a fc + /3 fc ) + ||pfe-i -p\\x~ 2^11^-1 ~yk\\ 2 
Adding the above inequality for k = 1, 2, . . . , n we conclude that 

1 n n 

^^ii Xfc - i_ ^n 2 - \\po-p\\\ + ^2( a k+Pk 

Therefore 



2M 

k=\ k=\ 



fc=i 

and, using also ([7|), we conclude that 



Using ([SD we have 



Therefore 



lim x k -i - Vk = lim a k + b k -i = 

k— >oo k— >oo 



lim y k -y k = lim a k - a k = 0. 

fc— »oo fc— >oo 



lim x fc _i - y fe = lim a fc + 6 fe _i = (12) 

k— >oo fc— >oo 



and sequence {{yk,bk)} is also bounded. 

Since the sequence {pk} is bounded and H x H (endowed with the norm 
|| • || A ) is reflexive, this sequence has weak cluster points. Let (x,b) be a 
weak cluster point of the bounded sequence {pk = (xk,bk)}- Using again the 
fact that H x H is reflexive, we conclude that there exists a subsequence 
{( x kj, bk 3 )} converging weakly to (x,b) and hence 

x kj -)■ x, b kj ->• b, as j -)■ oo, 

which, together with (112p implies also that 

y kj -\ -> x, a kj _i ->■ -b, as j -)■ oo. 

Using the two above equations, (JX2J) and Lemma[5](see Appendix|A]) applied 
to the subsequences {(xfc , b^}, {(y^-i, Ofcj-i}, we conclude that (x, b) £ B, 
(x,-b) e A, that is, (x,b) G 5(A,5). 

We have proved that the sequence {pk} has weak cluster points and 
that all these weak cluster points are in S(A,B). Using these results and 
Corollary [3] we conclude that {pk} has only one weak cluster point (x,b), 
and this (weak cluster) point belongs to S(A,B). As {pk} is bounded and 
H x H is reflexive, the sequence {pk} converges weakly to such point (x,b), 
which is equivalent 

Xk — >• x, bk —> b, as k — > oo, 



To end the proof, use the above equation and (|T2|) to conclude that 

yk — r x, ak —> —o, as k — > oo. 

□ 

The convergence analysis presented is are based on the framework and 
techniques introduced in [2J [5] , and becomes more intuitive using this frame- 
work and results. To make this note shorter, we do not presented this frame- 
work here. For historical reasons, here we used the classical sumable error 
tolerance. 

A An auxiliary result 

Let X be a real Banach space with topological dual X* . For x £ X, x* £ X* 
we use the notation (x, x*) = x*(x). An operator T : X =i X* is monotone 
if (x — y,x* — y*) > for all (x,x*), (y,y*) £ T, and is maximal monotone 
if it is monotone and maximal in the family of monotone operators with 
respect to the partial order of inclusion. 



Lemma 4. Let X be a real Banach space with topological dual X* . If 
T : X zj X is maximal monotone, {(xj, x*)}j g / is a net in T which converges 
in the weakxweak* topology to (x,x*), then 

lim inf (xi,x*) > {x,x*}. 

i— >oo 

Moreover, if the above inequality holds as an equality, then (x,x*) G T. 
Proof Let <p : X x X* -> R, 

(fix, x*) = sup (x, y*) + (y, x*) - (y, y*) 

The function <p is Fitzpatrick minimal function [4J of T, it is lower semicon- 
tinuous in the weakxweak* topology, (p(x,x*) > (x,x*) for all (x,x*) and 
this inequality holds as an equality if and only if (x,x*) G T. Therefore 
(xi,x*) = ip(xi,x*) for alii € I and 

lim inf (xi, x*) = lim inf tp(xi,x*) > ip(x,x*) > (x,x*) 

i— >oo ' i— >oo 

To end the proof, use the fact that </? is bounded below by the duality product 
and coincide with the duality product if and only if (x,x*) G T. □ 

Lemma 5. Let X be real Banach space. IfT\,...,T m : X =4 X* are 
maximal monotone operators and {(a?fc,j, x^. j)}ie/ are bounded nets such that 
{xk,i,x* k J G Tfc for all k = 1, . . . ,rn i G I, and 



Xk,i-Xj,i~+0 j, k = l,...,m y\ 



, x k,i ~^ x 
fc=i 
w _ *«>*_*, .. 

as i ^ co, then (x, x* k ) G T^ for k = 1, . . . , m. 
Proof. In view of the above assumptions, 

in 

E— * — * 

X fc =X 

fe=l 

Define 

«fc,i = (ajfe.i, art j) - (*> *fc) 3 k = l,...,mi £ I. 



Direct algebraic manipulations yield 

m / m \ 

fe=l \fe=0 / 

which ready implies, in view of the assumptions of the lemma, that 

in 
lim } a ki = 

fc=l 

Using the first part of Lemma [J] we have 

lim sup a k) i > 0, k = 1, . . . , m. 

i— >oo 

Combining the two above equations we conclude that lim^oo a kl i = for 
k = 1, . . . ,m, that is 

lim(x ktl ,x* ki ) = (x,x* k ) 

To end the proof, use the above equation and the second part of Lemma HI 

□ 
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